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A Video Editing System and Method 

This invention relates to the field of communications and 
information processing and in particular to the field of 
video editing. 

The advent of digital television and cable/satellite channels 
means the demand for edited moving image products is seeing a 
dramatic increase. At the same time, the use of digital 
video in many fields where traditionally film was used, and 
its widespread availability to non-professional users, 
induces many significant changes in the editing and creative 
processes involved in the making of a moving image product. 
These changes currently affect quality upstream TV 
production, domestic and semi-professional camcorder users 
and internet/www content. 

Video editing is, at present, slow and labour intensive. in 
this, the process of gathering and selecting video material 
is the main time consuming activity. Currently, selecting 
video content involves searching, often linearly, through raw 
video footage, identifying different blocks of footage 
("takes") and selecting and extracting into digital form a 
subset of the blocks. These are then used as building blocks 
for assembly into the final program. This process is only 
really manageable for reduced amounts of raw footage. 
Typically, however, a video production of two hours duration 
may utilise ten hours of raw footage. Much is therefore 
redundant (eg. different takes of the same scene) or not 
usable (e.g. when the camera has been left running during re- 
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positioning, in between shots, or images are of poor 
quality) . With such a large volume of material, editing 
becomes a time consuming, laborious and hence expensive 
process . 

5 

Currently there are many video editing packages available. 

These are aimed at the whole spectrum of home user to 

professional markets. The packages available include, for 

example, Adobe \ Dynamic Media Collection; Adobe! WebCollection; 
^t-m3 K <f«JM) ^TH) 

10 Sony jJVlo vie Shaker; Apple pFinal Cut Pro; Pinnacle "^Systems: 

Lightning 500, MicroVideo DV200, StudioMPIO; Fast Multimedia 

DV master pro. These provide an interface and conversion 

between various data formats. They generally cope with the 

challenges of editing compressed sequences with limited 

15 quality loss, e.g. MPEG motion compensated coding methods. 

They also enable special effects, e.g. slow motion, colour •* 

effects etc. However, known packages do not provide a 

suitable environment and set of tools to facilitate the 

material selection and arrangement process. Rather, they 

2 0 assume that this selection has been previously made and focus 

on later stages of video editing. 

With the coming of age of digital video processing, systems 
that can automatically segment and classify video shots have 
25 been proposed. Examples of such systems are described in EP 
0915469 A2; EP 0780844A2; US 6,195,458; US 5,635,982 and WO 
0103429. Other systems that have the ability to automatically 
search for specific kinds of shot have been developed. These 
are targeted at applications such as web browsing or 

3 0 interactive television or video on demand or automatic 
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editing, as described, for example, in WO 00/39707; WO 
99/41684, and EP 1143 714A1. A problem with known and 
proposed systems remains, however, that the basic editing 
platforms available are difficult to use, 

5 

JP200261757 describes a system/process to automatically 
remove unwanted material and select useful video sections. 
However, it does not allow for the intervention of the user. 
This limits the selection criteria to quantitative, 
10 computable ones, and hence overrides the creative human input 

• • • • 

in the selection process. 9m ** 9 

Various aspects of the invention are defined in the *** 
independent claims. Some preferred features are defined in 
15 the dependent claims. 

• t r 
« i 

According to one aspect of the present invention, there is 
provided a method for editing digital video footage using a 
computer processor, the method comprising: cutting the 
20 digital video footage into a plurality of clips and sorting 
the clips into groups based on selected criteria. 

By cutting up the video and allocating the resultant clips to 
groups that can be selected and opened by a user, the editing 

25 process is greatly simplified. This is because an editor can 
very quickly identify relevant clips merely from the groups. 
For example, the groups may comprise people, landscapes and 
weather. These simple categories enable the editor to 
readily locate and put together relevant shots. In contrast, 

3 0 existing packages require editors to scan footage linearly 
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and pick out clips of interest. Because of the length of 
typical footage runs, this is labour intensive and so, for 
the professional, costly. 

5 The method in which the invention is embodied may further 
involve searching for groups of clips based user input 
criteria. The method may also involve presenting on a screen 
or display a group identifier for each group of clips; 
receiving a user selection of a desired group identifier and 
10 presenting clip identifiers for clips included in the 

• • • 

selected group in response to the user selection. *-••< 

• • • 

* « 

The video data may be provided from multiple sources, for 
instance a video camera, a video tape, files in digital • 
15 storage media, etc. The video data of each clip may be in 
different formats. 
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The step of cutting the video data into clips may be done 
automatically using a cutting algorithm. As part of this 

20 process, the clips can be automatically classified into one 
or more different categories, relating to the quality of the 
images or camera motion parameters, for instance. The clips 
preferably have some uniform characteristics such as 
photometric characteristics or camera motion characteristics, 

25 but are not necessarily semantically significant. 

The step of sorting may comprise attributing, interactively 
or automatically, a set of label descriptors to each clip to 
facilitate a characteristics-based access and organisation of 
3 0 the video clips. The step of sorting may further comprise a 



o 

5 

stage of selecting a group of the video clips (which could be 
all of them) and organising/sorting them into one or a set of 
folders . 

5 The method may further involve a stage of automatically 
outputting and presenting the groups to a user in the form of 
a set of digital video clips arranged in a hierarchy of 
digital folders. 

10 The method may comprise presenting the video material 

• • • * 

gathered and segmented into clips as a virtual representation 

• * » • 

of the clips, organised according to their categories* • 
Within the categories, the clips may be further organised # » 
into classes expressed as labels or attributes. The method / 
15 may comprise receiving a user-defined classification for a 

* a 1 y 

clip and including a clip into a group based on the user 

• » * 

defined class. J 

The method may comprise adding labels/classes to some or all 

2 0 the clips, preferably in response to a user command. The 

method may involve overriding the labels/classes 
automatically attributed to some/all of the clips in response 
to an input received from a user. The method may involve 
toggling between different categories-based presentations of 
25 the clips, so as to use the classes in the different 
categories as an access criterion. The method may involve 
applying elementary modifications to the clips, such as 
merging two clips or splitting a clip in two. The method may 
further involve modifying the photometric attributes of the 

3 0 images in the clips. The method may comprise creating 
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hierarchies of virtual folders. The method may further 
comprise selecting some or all of the clips by virtually 
copying them into one or more of the virtual folders. 

5 The method may further comprise exporting the data to create 
automatically a directory structure containing digital video 
files on a digital storage medium, for example the hard disc 
of a computer. The structure and virtual content of the 
video directory as defined in the previous stages of the 
10 process is then mapped onto a digital physical folder 

• • • • 

structure containing physical digital files corresponding to "•••** 

• • . • 

the selected clips. • * # T 

• • • 

According to another aspect of the present invention, there • ... 

• • • 

• • • 

15 is provided a computer program, preferably on a data carrier 

* » # « 

or a computer readable medium, the computer program being for 

• • • 

editing digital video footage using a computer processor, the * 
computer program comprising code or instructions for cutting 
the digital video footage into a plurality of clips and 
20 sorting or organising the clips into groups based on selected 
criteria . 

The computer program may comprise instructions or code for 
searching for groups of clips based user input criteria. The 
25 computer program may also comprise instructions or code for 
presenting on a screen or display a group identifier for each 
group of clips; receiving a user selection of a desired group 
identifier and presenting clip identifiers for clips included 
in the selected group in response to the user selection. 

30 
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According to yet another aspect of the present invention, 
there is provided a system for editing digital video footage 
using a computer processor, the system comprising means for 
cutting the digital video footage into a plurality of clips 
5 and means for sorting or organising the clips into groups 
based on selected criteria. 

The system may comprise means for searching for groups of 
clips based user input criteria. The system may also 

10 comprise means for presenting on a screen or display a group 

• • • • 

identifier for each group of clips; means for receiving a 

• • * • 

user selection of a desired group identifier and means for *•! 
presenting clip identifiers for clips included in the 

selected group in response to the user selection. ^ 

• * * t 

15 

• * » » 

According to a still further aspect of the present invention, *.»»•• 
there is provided a digital camera or the like that includes J *».* 
a system or computer program in which the invention is' 
embodied . 

20 

According to a yet still further aspect of the present 
invention, there is provided video content that is produced 
using the method, computer program or system defined in the 
other aspects of this invention, or any copies of such video 

2 5 content. 

Various aspects of the invention will now be described by way 
of example only and with reference to the drawings, of which: 
Figure 1 is a flow chart of a method for pre-editing 

3 0 video footage; 
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Figure 2 is a more detailed flow diagram of the method 
of Figure 1 ; 

Figure 3 is a block diagram of a system for implementing 
the process of Figures 1 and 2; 
5 Figure 4 shows the degrees of freedom represented by a 

parameter M, M being used in the step of classifying of 
Figure 2 ; 

Figure 5 is a representation of three motion parameter 
domains used in the step of classifying of Figure 2; 
10 Figure 6 is a representation of image regions used in an 

• • • • 

algorithm for implementing the step of classifying of 

• • • * 

Figure 2; *.I 
Figure 7 shows a main screen of a user interface for .: # 

implementing the method of Figure 2; . * :** 

• <• • 

15 Figure 8 shows a screen that is presented when a user 

« * a * 

enters the user interface; 

Figure 9 is a screen that is presented when the "new : 
project" option is selected from the screen of Figure 8; 
Figure 10 is a screen that is presented when the 

2 0 "capture from source" option is selected from the screen 

of Figure 9 ; 

Figure 11 is a screen that is presented to allow the 
user to vary cutting or classification criteria, and 
Figure 12 is a screen that is presented to allow users 
25 to select cutting points, when trimming a clip. 

Figure 1 shows the steps in a method for editing video 
footage . These include receiving raw data from a video 
input; dividing the raw data into smaller video clips and 

3 0 sorting or allocating the clips into groups or sets based on 
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15 



20 



selected criteria. A more detailed flow diagram is shown in 
Figure 2, in which the step of allocating the clips to groups 
involves various stages of selection, classification and 
sorting. This will be discussed in more detail later. 

The methods of Figures 1 and 2 can be implemented using the 
arrangement of Figure 3. This has an interface 201 to the 
video data, which data may be obtained from different data 
input sources 200, such as a video camera, a video tape, 
files in digital storage media, etc. To deal with data in 
analogue form the interface may include a frame grabber, it ' 
could also have a firewire port if the data is on tape. 
Additionally or alternatively, a connection to a hard drive 
could be provided, if the data is in digital form. 



The video interface 201 of Figure 3 is able to communicate 
with user interface software 203 that is run by a processing :**.*• 
unit 202. The user interface software 203 is a computer 
program, which is adapted to implement the method of Figures 
1 and 2. Clips and other video content used or created by 
the interface software 201 are stored in a digital storage 
unit 204. As will be appreciated, the video editing platform 
of Figure 3 could be implemented in a standard PC or any such 
device or product with similar processing capability. More 
25 specifically, the platform could be implemented in a digital 
camera, with the video content being provided by the camera, 
or any other such consumer electronics device. 
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Returning to Figure 2, this shows various different steps of 
the process, starting from a data import stage when video 



10 



footage is introduced to an export stage when selected and 
sorted digital video clips are output. Any path outlined by 
the arrows in Figure 2 is possible between the import and 
export stage. The user interface software is adapted to 
5 allow the steps of Figure 2 to be tackled iteratively in a 
non- linear way, going back and forth between the different 
parts of the process as many times as required. However, for 
ease of explanation, each step of the process is described 
successively, in a linear way. 

10 

• • 4 • 

The first stage in the method of Figure 2 consists of 

• • * • 

assembling the raw video material that is to be edited. This *-J 
can be in the form of analogue or digital video format and 

can be provided on any storage medium, for example tape, 9 * 

• • • 

15 disc, memory, etc. This data is then digitised where 

• 9 * » 

necessary, and put in the form of a windows supported video *.»».* 
format. For analogue data the step of digitising can be done I \. m 
using an analogue frame grabber. The result of this stage is 
a representation of the raw video material as a set of 
2 0 digital video files on a digital storage medium. The files 
may be compressed or have a low resolution. These files 
constitute the basis of the representation of the raw video 
material for the overall process. 

25 During the import stage, enough information is kept in order 
to be able to re-iterate the import for all the used sources, 
producing the highest possible resolution for each of the 
digital files. The software is adapted so that all of the 
raw video material to be used for the final edited product 

30 can be imported at the start of the process, or alternatively 



new raw data can be added as needed at any stage of the 
process . 



Once the video content is digitised, the files are then 
divided or cut into a series of clips or sets of successive 
images that present certain homogeneity in one or more low- 
level image characteristics. This can be done using a 
histogram or statistical analysis of the data or by assessing 
the textural characteristics of the image or by looking at 
movement patterns, such as camera motion between frames. 
Cutting may be done automatically using a cutting algorithm 
included in the user interface software. In a preferred 
embodiment of the invention, two algorithms are used to 
perform cut detection. The first of these algorithms detects 
cuts by comparing histograms of intensity levels between 
frames; the second partitions the video by detecting the V 
appearance of intensity edges that are distant from edges inf 
the previous frame. 

Consider firstly the histogram algorithm. This compares the 
grey level histograms of consecutive frames D(i, . a cut 

is declared if the absolute sum of histogram differences 
between frames is greater than a threshold T, 

D(i,i + V) = ±\H i U)-H M Ui 

7=1 

where H ± (j) is the histogram value for the grey level j in 
the frame i, j is the grey value and n is the total number of 
grey levels (256 in the current implementation) . 
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The principle behind the histogram algorithm is that two 
frames having an unchanging background and unchanging objects 
show little difference in their respective histograms. 
Although there may be cases in which two images have similar 
histograms but different contents, the probability of such an 
event occurring in practice is sufficiently low that such 
errors can be tolerated. This is discussed in "Automatic 
Partitioning of Full -Motion Video" by Zhang et al , Multimedia 
Systems, 1993, pp. 10-28. 

As an alternative to the histogram algorithm, an intensity 
ecige algorithm can be used to determine where cuts in the 
video should occur. An example of this method is described 
in detail in the article "A Feature Based Algorithm for 
Detecting and Classifying Production Effects'' by Zabih et al, 
Multimedia Systems 1999, pp. 119-128. The intensity edge : 
method involves comparing the fraction/? of edges entering or 
exiting between two frames: p = max( p in ,p out ) , where p ±n measures 
the proportion of entering edge pixels and p out measures the 
proportion of exiting edge pixels. This represents the 
fraction of changed edges. This approach is based on the 
assumption that during a cut or a dissolve, new intensity 
edges appear far from the location of old edges. In a similar 
way, old edges disappear far from the location of new edges. 
Hence, by counting entering and exiting edge pixels, cuts can 
be detected. 
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The intensity edge algorithm takes two consecutive frames X 
and X' and firstly transforms colour images to grey images. 
Edge detection is then performed using the classic Canny 
algorithm. This algorithm is described in "Machine Vision" 
by R Jain et al, Mc Graw Hill International Editions, 
Computer Science Series, 1995. The Canny type edge detection 
produces binary images E and E' , in which edges are denoted 
by white pixels) . Next, copies of E and E' are created with 
the pixels dilated by a radius r, to form expanded images D 
and D' . Then the fractions of entering and exiting edge 
pixels are calculated. 

Consider p out the fraction of edge pixels in E which are 
farther than r away from an edge pixel in E' . A white pixel . 
E[x,y] is an exiting pixel when D' [x,y] is not a white pixel. 
This is because the white pixels in D' are exactly those " 
pixels within distance r of an edge pixel in E' . The equation* 
for p out is: 



This is the fraction of edge pixels that are exiting. p in is 
calculated similarly: 



Pout 



= 1- 



Pa 



in 



= 1- 
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Scene breaks are detected by looking for peaks in the edge 
change fraction />. When such peaks are identified, the video 
content can be divided. In this way, a stream of video can 
be divided into a plurality of clips, each presenting within 
5 itself a certain degree of homogeneity. 

Whilst histogram and intensity edge detection methods are 
described above, the process of dividing video into clips can 
be realized automatically, using various methods and 
10 algorithms. Indeed, as image processing algorithms become 

• * • • 

more sophisticated, semantically significant characteristics 

• * * • 

of the image could be used for the division of the material # * 
into clips. For example, algorithms could be used to detect .1. 

m 

the presence/absence of a talking head or a certain type of . # ;*: 

* * • 

15 object/creature or location-based characterisation or 
project-specific characteristics. Some suitable cut detection 
methods are presented in US 6,195,458, US 5,635,982 and US* 
5,339,166. Still further methods are described in the 
articles "Automatic partitioning of full -motion video" by 

20 Zhang et al , Multimedia Systems Vol. 1, 1993, pp. 10-28; 
"Temporal Video Segmentation: A Survey" by Koprinska et al , 
Signal Processing: Image Communication Vol. 16, 2001, pp. 
477-500 and "A feature -based algorithm for detecting and 
classifying production effects" by Zabih et al , Multimedia 

25 Systems Vol. 7, 1999, pp. 119-128. 

Once the video is divided, the system software is adapted to 
extract an internal representation of the clips as a pointer 
to a time index in one of the files extracted from the import 
3 0 phase, along with a duration or clip length. The 



representation and display of the clips in the other stages 
of the process is based on these pointers. The clip pointers 
or "virtual clips" are the basic units that further stages of 
the process manipulate. The end product of the process is a 
set of real digital clips corresponding to a subset of these 
digital clips, that is, a set of video files each containing 
at least one of the clips. 

In order to introduce flexibility and creativity into the 
cutting process, the user interface software is adapted to 
allow the automatic division into clips to be overridden by 
the end user. For example, the software may allow the user 
to split an existing clip into two or merge adjacent clips 
together. This means the user can choose to divide the data 
into clips according to higher level criteria such as 
semantic content . These video chunks are often referred to * 
as "scenes" . ; 

The process of Figure 2 further builds on the representation 
of the clips by attributing of a series of labels or classes 
to each clip. The clips can be classified in various ways, 
for example, according to clip start time and clip duration. 
This classification can be done automatically. Other 
classifications fall into a set of broad categories such as 
camera motion type; image quality criteria; user-defined 
classes, etc. In this case, the process of classifying is 
either done automatically into a set of pre-defined classes 
through the use of image processing classification algorithms 
or manually by the end user into a set of user classes that 
are defined dynamically. To allow for manual intervention, 



r 
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the user interface software is adapted to allow the user to 
override the classification output by the algorithms. In 
this way, the user can change any of the class labels of the 
clips as and when desired. 

5 

The classification algorithms could characterise the camera 
motion over the clip or the image quality, e.g. colour 
balance, contrast, blur/focus, etc. Algorithms for detecting 
scene changes and camera motion are known. Various examples 
10 are described in the articles "Efficient Scene Change 

• • • * 

Detection and Camera Motion Annotation for Video **•* 

• « • m 

Classification", by W. Xiong and J. C-M. Lee, Computer Vision 
and Image Understanding Vol. 71, PP 166-181; "A Unified .; # 

Approach to Shot Change Detection and Camera Motion # * 

• • • 

15 Characterisation", by P. Bouthemy, M. Gelgon and F. Ganansia, 

• • » • 

* * 

IEEE Transactions on Circuits and Systems for Video ••»• 

• • • 

Technology, Volume 9, No. 7, October 1999 and "The Robust* 
Estimation of Multiple Motions: Parametric and Piecewise- 
Smooth Flow Fields" by M.J. Black and P. Anandan, Computer 

20 Vision and Image Understanding, Volume 63, No. 1, January 
1996, pp. 75-104; "Out -of - focus blur estimation and 
restoration for digital auto- focusing system" by Kim et al, 
Electronic Letters, 1998, v. 34, n. 12, June. As regards 
blur detection, a suitable algorithm for implementing this is 

25 described in the article "Focusing" by E. 

Krotkov, International Journal of computer Vision, 1987, v. 
l,pp. 223-237. 

Whilst many different classifications are known and could be 
30 used, a preferred option is to classify all of the video 



clips into a motion label. This could, for example, be 
panning, tilt, rotation, zoom or even static. It could also 
be unrecognized, where the camera motion is chaotic, or 
complex, where the camera motion is defined by two or more 
simple motions. 

The preferred motion classification method uses Bouthemy's 
motion estimation. This estimates the motion between two 
consecutive frames, computing the values of a parametric or 
affine model (a 17 a 2/ a 3 , a 4 , a 5 , a 6 ) . Then, a new model M is 
defined from the previous one (resulting from a linear 
combination). In this case, M=(m 1 , m 2 , m 3 , m 4 , m 5/ m 6 ) , where 
m ± = {0, >0, <0} 7 i<= (1...6) and each of these parameters 
represents an easy physically meaningful interpretation of • 
the dominant motion: m x - horizontal translation; m 2 

« 

forwards or backwards translation; m 4 - vertical translation; 
m 5 - rotational motion and m 3 , m 6 - hyperbolic terms. Figure* 
4 shows a graphical representation of these parameters . 

At each instant t, the method interprets the value of the 
vector M and converts it into a motion label. This operation 
has to be repeated over all the frames of the clip so as to 
determine the global motion. Due to noise, estimation 
errors, and the use of an approximate motion model, the 
components of the vector M cannot be exactly equal to 0, even 
if it should be the case. The aim is then to decide whether 
these estimated values are significant or not. To do this, 
the following likelihood log-ratio test is performed: 



18 



r t = In 



/oO/) 
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> 

Ha 



X with ie (1...6) 



where / 0 (#O is the likelihood function for m i - 0 and ^(m,) 
the likelihood function for m f ^0. If the ratio is lower 
than the threshold X , the component m ± is declared to be 
significant, otherwise it is considered to be null. 

The threshold X is determined by the value of mintri) . In 
fact, it is assumed that the most relevant parameter is the 
one that has a minimum log-ratio. Hence, parameters having a 
log-ratio less than a percentage of the minimum log-ratio are 
not significant. For the purposes of the present method, 5% 
is deemed to be a good compromise. Hence, the threshold X is 
defined as 5% of the value of min(r ± ) . 

The above estimation algorithm is relatively slow, i.e. not 
real-time. Because of this, a small frame resolution 

(160*120) is used and the estimation parameters are set to 
the smallest value. In this way, the processing time can be 
speeded up. However, a disadvantage of this is that the 
significance of one or many parameters modifying the type of 
global motion may be artificially increased, thereby making 
some otherwise insignificant parameters seem dominant. This 
issue can be partially addressed by using a second threshold 
y to enable the most dominant parameters to be distinguished 
from parameters that are less dominant, but not 
insignificant. These less dominant parameters are referred 
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to as being influential parameters and are defined as having 
a log-ratio of between 5 and 20% of minCr^ . The boundaries 

between the dominant, influential and insignificant 

parameters are shown graphically in Figure 5 . 

Although not as significant as the dominant parameters, the 
influential parameters themselves may include some real 
information as well as information that may have been 
produced erroneously. For example, in a video clip where 
something is moving around (like people in a store) , the 
influential parameters describe this type of secondary 
motion. Hence, influential parameters actually provide 

information about the scene and what happens inside it. So, 
in some cases, it would be useful to interpret these • 
parameters. The difficulty is however differentiating the 
relevant information in the influential parameters from the * 
noise. To do this a parameter representation model is used. ' 

The parameter representation model uses six parameters (Model 
M) which can be positive, negative or null. Now, a new model 
is defined to represent them in a binary system. This is a 
very simple way to store information on each parameter. The 
model is defined as follows: 



V 2 , \y 3t \j/ 4 , vj/ 5 , xf/ 6 , vj/ 7 , v|/ 8/ v „ Vl0 , Via) 

with Vl = {0, >0}, Ie(U6) and Vl = {0, <o}, Ie (7...12) . Using 
this ' a panning left is characterized by 

(0,0,0,0,0,0,1,0,0,0,0,0), whereas a panning right is 

(1,0,0,0,0,0,0,0,0,0,0,0) . 



20 



10 



20 



25 



In the present case, two models are created. The first one 
represents the dominant parameters and the second one 
represents the influential parameters associated with the 
dominant parameters. Each of the two provides a vector, one 
describing the dominant parameters and another representing 
the influential parameters. To determine the dominant motion 
between two frames, only the dominant parameter vector is 
used. This gives an indication of the global motion. This 
step is repeated for all the frames of video in order to 
obtain a global picture of the motion. 



4 

* » 

• » 



To determine the motion within a video clip, information on # * 

• • • 

every second frame is stored. This information is dominant 
15 motion parameter, influential motion parameter, and time 

parameter. The last parameter gives the time position of the ! '.»* 
frame within the clip in seconds. To store this information 
a list of records is created, this list representing the 
dominant motions appearing in the video clip. Each record has 
a counter that is incremented by a unit every time 
information obtained matches with one of the records created. 
To determine whether matches exist, time motion information 
from two consecutive frames is obtained and both the dominant 
motion and time parameters are compared to those already 
existing from the list record. A match is defined as 
occurring when the dominant motion parameter is the same as 
the one from the record and the time parameter is less than 
ten times the record time parameter. In the event that there 
is a match, the relevant counter is incremented by one unit 
and the existing time parameter is refreshed or up-dated with 
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the new time parameter. If there is no match, a new record 
is created for the motion at the end of the list. Once this 
is done for all of the frames of the clip, if the list yields 
a motion appearing in more than 5 0% of the number of video 
5 frames, then this motion is considered as the dominant one. 
Otherwise, it is assumed that the video clip has no global 
motion. 

Another method of classifying video clips uses optical flow 

• • * * 

10 techniques. To compute the optical flow, the very well known 

• • • « 

Lucas and Kanade's algorithm is used. This is described in 
detail in the textbook "Introductory Techniques for 3D # ; # 
computer vision" Emmanuele Trucco, Alessandro Verri Prentice 

• • • 

Hall. A basic assumption in computing optical flow is that 

• a * * 

15 the brightness of every point of a moving or static object *•»*»* 
does not change over time. In order to determine optical • *•»* 
flow, each frame of the video is divided into four regions, 
as shown in Figure 6. Once this is done, the number of 

pixels not moving is counted. If more than 2 0% of the pixels 

2 0 are moving then it is assumed that there is camera motion on 

the scene. If more than 80% of the pixels are not moving then 
it is assumed that there is no camera motion on the scene. If 
there is camera motion, the direction of the optical flow is 
analysed by calculating the mean and standard deviation of 
25 the motion vectors. If the standard deviation is smaller than 
a fixed threshold x then it is either a pan or a tilt and the 
direction is determined by the mean direction of the optical 
flow vectors. Otherwise, if the standard deviation is bigger 
than x, the motion is either a zoom or a rotation and the 

3 0 mean in the regions is used to determine if it is a zoom 
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out/in or the direction of the rotation. The idea behind this 
is that on a pan or tilt most of the pixels are moving in one 
direction, thus the standard deviation is small. In 
contrast, in a zoom or rotation, pixels are moving in 
different directions and therefore the standard deviation is 
bigger. 

After finding the type of motion, the average direction of 
the optical flow is used to determine the direction of the 
motion. In the case of pan and tilt, the average direction of 
the optical flow in the whole image is used. This is because 
most of the pixels should be moving in the same direction. 
For the zoom and tilt, the average direction of the optical 
flow is compared in different regions according to observed 
patterns. For example, in the case of a zoom out 0iv > Qm> &n > 
9i, 0i <= (-7U, tc) 3 where Q t is the average direction of the optical 
flow on region i. It should be noted that the condition 9 IV > 
G x is unique for the zoom out . 

Q t and a are the average of the direction of the optical flow 
on region i and standard deviation of the direction of the 
optical flow on region i respectively. When no index is 
specified they denote the average and standard deviation in 
the whole image, respectively. The classification criteria 
are then set so that if a > % then the motion is a pan or 
tilt. In this case, 9 is used to determine the direction of 
the motion. Otherwise, camera motion is determined to be a 
zoom or rotation. More specifically, if 8 IV > 9 X , then it is 
a zoom out; if Q T1 > 0 IV , then it is a zoom in; if Q m > 0 
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then it is a rotation anti-clockwise and if Q T > Q xx , then it 
is a rotation clockwise. In the current implementation, 
every nineth frame is analysed, although, the software may be 
operable to allow the user to vary this. To consider a scene 
5 to constitute a category or class at least 50% of the 
analysed frames must be of the same category, otherwise it 
would be considered a x Complex Motion'. 



10 



In addition to motion detection, the clips can be sorted 

• * * • 

according to whether or not they are blurred or the level of 

• • • * 

contrast. As regards blurring, this can be detected using *** 

various different techniques. As a preferred option, blurring 

is detected using Canny's edge detection algorithm. As •* "I 

• • • 

mentioned previously the Canny algorithm is very well known. 

• * * « 

15 The blur detection is based on the principle that if the 
detected edges are not well defined then the image is blurred* 
and if the edges are well defined the image is not blurred. 
Experimentally, it was found this is a useful measure of how 
to predict levels of image blurring. 
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Contrast detection can also be used as a means for 
classifying clips. This can be useful because video 

sequences often include frames that are white or black, 
indicating that there has been a cut or change of scene. In 
25 some other cases detail cannot be detected, because frames 
are too bright or too dark. The reason of lack of contrast, 
in other words the existence of image frames that are "too 
bright" or -too dark-, is that pixels within the image that 
represent the objects or part of objects tend to have 
intensity values which are very similar/close to each other. 
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This can be observed in a histogram of the intensity values 
of the image. The histogram of a poorly contrasted image is 
very narrow, in other words the pixel values are clustered in 
a narrow area of the histogram, while the histogram of an 
image with better levels of contrast would be more spread 
out . 

The histogram of an image is a discrete function that is 
formed by counting the number of pixels in the image that 
have certain intensity values. When this function is ** 

normalised to sum up to 1 for all the intensity level values, 
it can be treated as a probability density function that # 
expresses the probability of a certain intensity value being # » 
found in the image. Seen this way, the intensity value of a 
pixel is a random variable that takes values according to the *« 

C f 

outcome of an underlying random experiment. Ideally, a well- I 
equalised image would have a flat-ish histogram. In theory, 
it would then be possible to enhance an image to have a 
relatively flat histogram, so that the pixels can be randomly 
re-distributed across neighbouring intensity values. 

Since the aim here is to detect and classify the lack of 
contrast in an image, looking at the intensity values of the 
pixels as a random variable is useful. As mentioned before, 
a good distribution of pixel intensity values is indicative 
of a well -contrasted image. Hence, this distribution could be 
measured through the standard deviation of the intensity 
l eve l s . if the standard deviation is high, a wide 
distribution of the intensity pixel values is present, 



implying good contrast. Otherwise the image has poor contrast 
levels . 



In order to distinguish between good and poor levels of 
contrast, thresholds are set. A flat histogram was assumed 
as an example of a well -contrasted image in order to set the 
standard deviation threshold. The histogram was divided in 
three areas, corresponding to the first 2 0%, the next 6 0% and 
the last 20% of the bins. The first 20% of the bins are 
associated with dark intensity values, the next 60% are' 

< 

indicative of medium intensity values and last 20% are bright 
intensity values. Experimentally it was found that if the 
standard deviation of the intensity values is less than 4, a. 
black or a white frame is present. To discriminate between 
black and white, the intensity values are checked to* 
determine the area of the histogram they belong. if more 1 
than 5 0% of the intensity values are in what was defined the 
dark area, a dark frame is present. If more than 50% of the 
intensity values are in the bright area of the histogram, a 
bright frame is present. In this way frames that are "too 
dark" or "too bright" can be detected. 

Whilst the methods described above are preferred, the process 
of classifying the clips can be realized automatically, using 
various methods and algorithms. For example, the clips could 
be classified on the basis of semantically significant 
criteria or whether or not there is a "talking head" in the 
clip . 
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Once the clips are classified according to whichever 

classification method is used, a sorting process is 

conducted. Based on the class labels attributed to the 

clips, the user interface software automatically presents the 

5 clips to the end user in an organised form. Typically, the 

clips are arranged in an array of folders, each folder 

representing a particular classification. The end user is 

then able to repeatedly select amongst different sorting 

criteria, which correspond to the previously mentioned 

10 categories, such as: camera motion type; image quality 9 

• * 

criteria; user-defined classes; clip start time; clip 
duration, etc. The clips are then presented in an organised 
way where all clips with the same label within one category/^*: 
are grouped together. This automatic sorting facilitates 
15 characteristics-based access to the material for the 
selection. 

Having cut, classified and sorted the clips, the video can be 
readily selected for use. This aspect of the process is 

20 realised interactively by the end user. The present invention 
facilitates this stage through a suitable virtual 
representation of the video clips and of the storage 
structure in which they are to be organized, and through 
enabling a suitable level of interactivity. The virtual 

25 representations are presented in groups, according to the 
clip classifications, via a software generated user 
interface. This will be described in more detail later. The 
user can then select some or all of the virtual clips and 
arrange them into a hierarchy of folders that he defines 

30 himself. 



* c * 

* • « 

* * * 
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The last stage of the process involves exporting the clips 
selected by the user. To do this, digital files are created 
on a digital storage medium, organised in a real directory 
5 structure so as to map the virtual representation of the 
video clips and of the storage structure established during 
the previous stages of the process. The video files are 
extracted from the already digitised, possibly low-resolution 
version of the video data used as a basis for the clip 

• * A % 

10 segmentation and classification in the previous stages. 

• ♦ • • 

Alternatively, they can be extracted from the original video *•• 
sources, that would then where relevant be recaptured at the 

requested resolution. In this way, the method provides/ 

• • • 

edited video content as selected by the user. Of course, this 

• * * • 

15 video content could be processed further to improve quality" 1 a " 
and/or copied to provide multiple copies for distribution or* *•»* 
otherwise . 



In order to allow a user to edit video and/or access pre- 
2 0 existing video clips, a user video interface is provided. 
The interface software is aimed at taking the user through 
the steps required to add video into the system in a rapid 
and friendly manner. The main interface screen is shown in 
Figure 7. However, the software is operable to present a 
25 wizard upon entry into the system and/or when a new project 
is being created. The first wizard screen is shown in Figure 
8 and is titled "Wizard Step 1 of 3". This provides the user 
with two selectable options, the first being to start a new 
project and the second being to open an existing project. 
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If the existing project option is selected and the "next" 
button clicked, the screen of Figure 7 is presented to the 
user. This will be discussed in more detail later. If the 
new project option is selected and the next button is 
clicked, the screen of Figure 9, titled "Wizard 2 of 3", is 
presented. This allows two further options, firstly the 
capture of video from a source and secondly the addition of 
an existing video file. If the capture from source option is 
selected, a list of all available Windows video capture 
devices is presented. When the Firewire (IEE-1394, iLink) 
device is selected then various options to remotely control 
the device are presented. Once the relevant device is 
selected, the "next" button is clicked and the user is f 
presented with the screen shown in Figure 10. 

« 
« 

Figure 10 shows the selected clip in the main view screen and * 
provides various remote control options that are selectable 
via a series of interactive buttons. These buttons include 
play, pause, record, stop, rewind, fast -forward and eject (if 
available) . Figure 10 also shows a data entry field for 
allowing the name of the tape or source medium to be input . 
This sets up an explorer style clip folder structure. If the 
eject option is available then it resets the name of the tape 
when ejected. Also provided is an option to capture at lower 
resolution (half) and run all the editing/clip selection on 
that and then batch capture all the necessary clips in full 
resolution at the output stage. If video is captured at low 
resolution, the software remembers this so at the output 
stage the video may be automatically recaptured at full 
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resolution by the software. 

Selection of the record button of Figure 10 causes an 
interactive "save dialogue" pop-up box to be displayed (not 
5 shown) . This "save dialog" box allows the user to select 
where to store the captured file and what name to give to the 
file. There is a default file name, which is used in the 
event that the user does not provide an alternative. The 
dialog remembers what directory it was last initiated in so 
10 that on future instances the user does not have to navigate 
back to the directory. Once the save parameters are entered 
by the user and received by the software, the "save dialogue" 
box is exited and the software automatically triggers the.*";*; 
recording of the video content on the digital storage unit 
15 204 of the system of Figure 3. This recording is done at the'""' 
selected resolution for the device (camera, VCR etc) . AfterS***-* 
the video has been captured the selected file name is added 
to a listbox in the screen of Figure 9, e.g. as files video 
1, video 2, video 3 etc. 

20 

Returning now to Figure 9, in the event that the "Add video 
from existing files" option is selected, the standard Windows 
open file dialog is presented. After the "Ok" button is 
pressed in the open file dialog, the selected file is added 
25 to the listbox shown in Figure 9. A file from the listbox 
can be removed if it has been added by accident . 

Once the desired video is selected using the screen of Figure 
9, the next button is selected. This causes the screen of 
3 0 Figure 11, titled "Wizard Step 3 of 3" to be presented. This 



30 



allows certain aspects of the functionality to be customised 
by the user, such as the type of cut detection or 
classification required. Algorithms are integrated into the 
software as plug- ins, and therefore the set of available 
5 classifiers and cut detectors can be upgraded or modified 
very easily. 

Figure 11 has three selection fields. The first allows the 

selection of desired video footage from the list of video #w 

« * 

10 files that were chosen by the user when in the screen of \ 

Figure 9. Another of the fields lists the basis on which the 

selected video is to be cut, for example on the detection of 

motion blur; focus blur and sudden scene change. Selection*^ 1*2 

of one or more of these options determines the algorithms ^ 

* * 

15 that are to be used to carry out the stage of cutting the^' 1 * 
video footage into clips. As described with reference to* 
Figures 1 and 2, the cut detection algorithms could use a 
histogram or an intensity edge approach. However, many 
different algorithms could be added either to qualify the 

2 0 image quality (color balance, brightness...) or the image 
content at a lower or higher level of understanding ("moving 
objects in image", "head and shoulder", nandscape"...) . 

The last data entry field of Figure 11 presents various 
25 options for the step of classifying the clips. In the screen 
shown, the option "pan direction" is selected. Other options 
such as motion type or image quality as previously described 
could of course be presented to the user for selection. In 
this way, the user is able to choose a particular type of 
30 clip classification as desired. 



When the user has made all of the relevant selections, the 
"finish" button is clicked. The user selections are then 
received by the interface software and interpreted and the 
relevant algorithms are run. The cut algorithm is run first 
to divide the video into clips and then the classification 
algorithms are run to act on the clips obtained after 
completion of the cutting stage. To give a visual indication 
to the user that work is being done, a progress bar is 
provided. After the step of classifying is completed, the 
clips can be sorted either automatically or in response to a 
user selection. The groups can then be presented to the user 
for selection and viewing. The main display of Figure 7 is 
used to allow users to select sort criteria, present clips 
and select those clips for export. 

Figure 7 shows the main system display. This has a file menu 
3 01; a preview window 3 03; a sort selection window 3 02; a 
timeline 304; a clip labeller 308; a clip icon display 310; a 
trash can 311 and a clip explorer 312-313. 

The file menu 3 01 has various options that are displayed when 
the "File" button is selected. These options include new; 
open; save; save as; add new clips; export and exit. The 
"New" option clears out the current project and starts a new 
one. The "Open" option clears out the current project and 
shows the standard Windows open file dialog. "Save" saves 
the current project using purpose built format. If the 
project has not been saved before then it will be saved. 
"Save as" shows the standard Windows save dialog box and if 
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selected saves the project using the using the purpose-built 
format. "Add new clips" displays the wizard of Figure 9 so 
that the user can go through the previously described steps 
for adding content. It should be noted, however, that in 
5 this case, the wizard title is changed to u ... Step 1 of 2" 
and the "previous" button is removed. "Export" takes the 
virtual explorer folders and replicates that structure on 
disk. If the clips have been captured at low resolution then 
this provides the option of batch capturing at full 
10 resolution. "Exit" exits the application. If the project is 
unsaved or changed then a prompt is generated to ask if the 
user wants to save the project. 

The preview window 3 03 is the gateway between the main 
15 display and the video viewer/editor window where adjustment 
of cut location may be done. Clips can be dragged from the 
timeline 304, or the clip explorer 312-313 or the clip sorter 
3 09 to this window. They are then automatically played in a 
loop. After the first loop has been completed the video is 

2 0 returned to the start and paused. When the user clicks once 

on this window the player alternates between pausing and 
playing the video. Double clicking opens up the video 
viewer/editor window. This is shown in Figure 12. 

25 The screen of Figure 12 has a main display 501 for viewing 
video clips and a bar 502 with a selectively movable marker 
that can be used to select the cutting points in the video. 
Movement of the bar marker causes movement of the image on 
screen. To select the start of the desired clip the software 

3 0 is operable to respond to receipt of a signal generated when 
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the left button on the user's mouse is clicked. When such a 
signal is received the start position is set at the location 
corresponding to the position of the marker bar. To select 
the end of the desired clip, the software is operable to 
5 respond to receipt of a signal generated when the right 
button on the user's mouse is clicked. In this case, when a 
right click signal is received the stop position is set at 
the location corresponding to the position of the marker bar. 
In this way the length of the clip can be defined. Also 
10 provided on the screen of Figure 12 are a browse bar 5 03 and 
a jog control 5 05 that allow the user to browse through the 
video footage. As these controls are varied, the video 
displayed on display 501 changes accordingly. Function 
buttons 506 are optionally provided to allow the user to 
15 manipulate the image. These buttons could, for example, 
allow the contrast to be enhanced or image balance corrected. 

Returning now to Figure 7, the sort selection window 3 02 
provides various options to allow a user to select the type 

2 0 of sort that is to be used to display all the data in the 
clip sorter 310. For example, the clips can be sorted by 
time. In the event that this is the user selection, the 
software searches through the video clips and groups them 
according to time section, which time section may or may not 

25 be determined by the user. In this case, each tab sheet 3 09 
in the clip sorter 310 then corresponds to the selected time 
sections of the data. If there are multiple video files then 
each tab sheet in the clip sorter 3 09 is the clip name and 
each one of these tabs has sub-tabs divided as before. 

30 
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The clips can also be sorted according to the automatic 
classifications. This can be done using the class labels 
attributed to the clips in the earlier classification stage. 
If the automatic option is selected, the user interface 
software automatically looks for the default classifications, 
searches for clips having class labels associated with these 
and presents the clips found to the end user in an organised 
form. Typically, the clips are arranged in an array of 
folders, each folder representing a particular classification 
such as: camera motion type; image quality criteria; contrast 
levels etc. The clips are then presented in an organised way 
where all clips with the same label within one category are 
grouped together. In this case, each tab sheet in the clip 
sorter 3 09 corresponds to one of the categories that is 
output by the automatic classifiers. The categories could 
include "Fixed shots", "Panning" , "Zooming", "Other", etc. 
Sorting may also be done by image quality. For example, each 
tab sheet in the clip sorter 309 may correspond to categories 
such as "Dark", "Light", "Poor contrast", "Blurred", "shaky", 
"OK". Further sorting criteria may be based on clip/shot 
duration or user categories. In the case of sorting by 
duration, each tab sheet in the clip sorter corresponds to 
ranges of duration for the clips, e.g.: "from 1 to 2 
minutes" . 

For sorting by user categories, the software is adapted to 
receive and store user attributed classifications associated 
with the clips, which user classifications are manually 
entered. To allow this, the software is operable to allow 
users to customise their own category list, e.g. "Landscape", 



"Static Landscape", "People", "Long shot", "Close-up". 
Visual inspection of the clips allows users to allocate 
specific clips to their own customised categories. Of course 
each clip may be allocated to a plurality of different 
categories . 



Other sorting options are possible. For example window 3 02 
could include an image quality option. In this case, the 
interface software would search for all clips that have been 
classified with an image quality label and group the clips 
together accordingly. of course, as will be appreciated, in 
this case the clips that are found could have been 
automatically or manually classified by the user. As before, 
each tab sheet in the clip sorter would then correspond to 
categories such as "Dark", "Light", "Poor contrast", 
"Blurred", "shaky", "OK". 

The timeline 3 04 shows the cut location for the clips 
relative to their position in the video. In this, selected 
clips are highlighted and below the clip a colour-coded 
classification 306 is provided, this being based on the 
automatic classification. in the bottom right of the 
timeline is provided an indication of whether the clip has 
been moved to the clip-explorer 313. Clips that are deemed 
unsuitable are hashed over. This is also done to clips that 
have been placed in the trash-can 311. 



Clips in the timeline 304 or the clip icon display 310 can be 
"dragged and dropped" to the preview window 3 03 or Trash Can 
311. Single clicking on a clip highlights it and shows it in 
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the preview window 303, thereby allowing the user to watch at 
least a segment of the clip. Single clicking also causes the 
clip data to be shown in the clip labeller 308. In addition, 
a go to tab and clip within that tab is shown in the clip 
5 sorter 309-310. If a clip is selected from the timeline 304 
and the shift key is held down and an adjacent clip is 
clicked on, then the two clips are selected and the merge 
clip button 3 07 is enabled. Clicking on this button 307 
merges the clips together. The data of the first clip in time 
10 order is then used as the new clip data. 

The clip icon display 310 should be the same as in the 
timeline 3 04 with the addition of clip info below the icon. 
The clip sorter displays all the clips according to the 

15 sorting criteria selected in the type of sort 3 02. It is 

assumed that the end user is familiar with the raw material 
and will be using this feature for rapid shot selection, e.g. 
"I need a static landscape shot". The clips can then be 
sorted by automatic classification or if they have labelled 

20 the shot they can sort by user categories. The clip sorter 
supports drag-n-drop to the clip explorer 312-313 or preview 
window 303 or trash can 311. Single clicking on a clip icon 
highlights it and shows it in the preview window 303 . This 
also causes the clip data to be shown in the clip labeller 

25 308 and highlights the clip in the timeline 304. 

The clip explorer 312-313 provides a Windows explorer 
presentation of clips to enable user to create a virtual clip 
folder hierarchy. It also provides the ability to delete 
3 0 named virtual folders and to drag-n-drop clips from clip view 



to trash can 311 and to preview window 303. Single clicking 
on a clip highlights it and shows it in the preview window 
303. This also shows the clip data in the clip labeller 308 
and causes the clip to be displayed and highlighted in the 
timeline 304 and clip sorter 310. 

To enable files to be added to the clip explorer 312-313, the 
user interface software is operable to allow each of the 
sorted clips 3 05 shown in the timeline 3 04 or the clip icon 
display 310 to be selected by the user. This is done by 
allowing the clip icons to be w dragged -and -dropped" into the 
clip explorer 312-313. Each selected file is added to a 
virtual file of selected clips. The order of the clips can 
be determined be the user. When all the desired clips are 
chosen and added to a virtual folder, the "Export" option in •* 
the file menu is selected. This takes the virtual file and ** 
replicates that structure on disk. This causes the automatic 
creation of a video directory structure for containing 
digital video files. This can be stored on, for example the 
hard disc of the user's computer. The structure and virtual 
content of the video directory is then mapped onto a digital •* 
physical folder structure and the physical digital files 
corresponding to the selected clips are retrieved and stored 
in the physical folder structure. At this stage, if the 
clips were captured at low resolution then the Export option 
allows batch capturing of the selected clips at full 
resolution. In this way, edited video content is provided. 

The user interface of Figures 7 to 12 provides a simple and 
effective interactive platform for editing video content in 
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accordance with the methods of Figures 1 and 2. As will be 
appreciated, various data inputs and video file formats could 
be used. For example, DV Video and other supported windows 
video capture devices; MPEG; M JPEG ; other Windows AVI 
5 supported file formats and previously processed video 
content. Once the content is received it is divided into 
clips. The step of dividing can be done automatically on the 
basis of default criteria. However, to improve flexibility 
and functionality for the user, the interface software is 
10 . operable to interactively receive criteria selected by the 
user and use these to determine where the video should be 
cut. In the event that trimming of the clips has to be done 
or the user wishes to over-ride the automatic cutting, an 
editing screen is provided to allow manual manipulation of 

» « • • 

15 the cutting process. The software is also operable to *•*»»' 

• * » * 

classify the clips. Again, the classification can be done on 
the basis of default or user determined criteria, such as 

* 

camera motion or blurring etc. Once the video is cut and • 
classified, the clips are sorted into groups based either on 

• » • * 

20 the selected classifications or on other user entered 

mm • 

criteria. Each group of clips can be presented on screen, so t *%• 
that a user can very quickly and easily find what he is 
looking for. In this way, it can be seen that the platform 
in which the present invention is embodied provides a simple, 

2 5 interactive and fast mechanism for editing video content. 

A skilled person will appreciate that variations of the 
disclosed arrangements are possible without departing form 
the invention. Accordingly, the above description of a 

3 0 specific embodiment is made by way of example only and not 



for the purposes of limitation. It will be clear to the 
skilled person that minor modifications may be made without 
significant changes to the operation described above. 
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Claims 

1. A system for editing digital video footage using a 
computer processor, the system comprising cutting means 
for cutting the digital video footage into a plurality 
of clips and sorting means for sorting the clips into 
groups * 

2. A system as claimed in claim 1, comprising means for 
presenting on a screen or display a group identifier for 
each group of clips; means for receiving a user 
selection of a desired group identifier and means for 
presenting clip identifiers for clips or clips included 
in the selected group in response to the user selection. 

3 . A system as claimed in claim 1 or clam 2 comprising 
means for searching for groups of clips based on user 
input criteria. 

4. A system as claimed in any one of the preceding claims, 
wherein the means for cutting are operable to cut the 
video footage automatically using a cutting algorithm. 

5. A system as claimed in any one of the preceding claims, 
wherein the cutting means are operable to detect changes 
between frames of the video and define a cut when the 
changes exceed a pre -determined level. 

6. A system as claimed in any one of the preceding claims, 
wherein the cutting means are operable to cut the video 



content into clips on the basis of an intensity edge 
determination. 



A system as claimed in any one of the preceding claims, 
wherein the cutting means comprises a histogram 
algorithm. 

A system as claimed in any one of the preceding claims, 
wherein the cutting means are operable to cut the video 
content into clips in response to a user command. 

A system as claimed in any one of the preceding claims 
comprising means for manipulating clips in response to a 
user command. 

A system as claimed in claim 8, wherein the means for 
manipulating are operable to merge selected clips in 
response to a user command. 



A system as claimed in claim 8, wherein the means for * 
manipulating are operable to divide a selected clip in i 
response to a user command. 



A system as claimed in any one of the preceding claims 
comprising means for classifying the clips according to 
pre-determined classification criteria. 

A system as claimed in claim 11, wherein the classifying 
means are operable to classify the clips according to 
any one or more of camera motion; optical flow; image 
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quality; scene changes; photometric characteristics; 
level of blurring and contrast detection . 

A system as claimed in claim 11 or claim 12, wherein the 
classifying means are operable to classify a clip in 
response to a user defined classification, 

A system as claimed in any one of claims 12 to 14, 
wherein the sorting means are operable to sort the clips 
into groups based on clip classifications. 

A system as claimed in any one of the preceding claims 
comprising means for attributing a label descriptor to 
each clip. 

A system as claimed in claim 16, wherein the attributing 
means are operable to automatically attribute a label 
descriptor to a clip. 

A system as claimed in claim 16, wherein the attributing 
means are operable to attribute a label descriptor to a 
clip in response to a user selection. 

A system as claimed in any one of the preceding claims, 
wherein the means for sorting are operable to sort the 
clips into one or a set of folders. 

A system as claimed in any one of the preceding claims, 
wherein the means for sorting are operable to include 
each clip in a plurality of different groups. 
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A system as claimed in any one of the preceding claims, 
wherein the means for sorting are operable to sort the 
clips in response to a user selection * 

A computer program, preferably on a data carrier or a 
computer readable medium, the computer program being for 
editing digital video footage using a computer 
processor, the computer program comprising code or 
instructions for cutting the digital video footage into 
a plurality of clips and sorting the clips into a 
plurality of groups. 

A computer program as claimed in claim 22, comprising 
code or instructions for presenting on a screen or 
display a group identifier for each group of clips; 
receiving a user selection of a desired group identifier 
and presenting clip identifiers for clips included in 
the selected group in response to the user selection. 

A computer program as claimed in claim 22 or claim 23, 
comprising code or instructions for searching for groups 
of clips based on user input criteria. 

A computer program as claimed in any one of claims 22 to 
23, wherein the code or instructions for cutting are 
operable to cut the video footage automatically using a 
cutting algorithm. 



44 



A computer program as claimed in any one of claims 22 to 

25, wherein the code or instructions for cutting are 
operable to detect changes between frames of the video 
and define a cut when the changes exceed a pre- 
determined level . 

A computer program as claimed in any one of claims 22 to 

26, wherein the code or instructions for cutting are 
operable to cut the video content into clips on the 
basis of an intensity edge determination. 

A computer program as claimed in any one of claims 22 to 

27, wherein the code or instructions for cutting 
comprises a histogram algorithm . 

A computer program as claimed in any one of claims 22 to 

28, wherein the code or instructions for cutting are 
operable to cut the video content into clips in response 
to a user command. 

A computer program as claimed in any one of claims 2 2 to 
2 9 comprising code or instructions for manipulating or 
modifying clips in response to a user command. 

A computer program as claimed in claim 30, wherein the 
code or instructions for manipulating are operable to 
merge selected clips in response to a user command. 



A computer program as claimed in claim 30, wherein the 
code or instructions for manipulating are operable to 
divide a selected clip in response to a user command. 

A computer program as claimed in any one of claims 22 to 
33 comprising code or instructions for classifying the 
clips according to pre -determined classification 
criteria . 

A computer program as claimed in claim 33, wherein the 
code or instructions for classifying are operable to 
classify the clips according to any one or more of 
camera motion detection; optical flow; image quality; 
scene changes; photometric characteristics; level of 
blurring and contrast detection. 

A computer program as claimed in claim 33 or claim 34, 
wherein the code or instructions for classifying are 
operable to classify a clip in response to a user 
defined classification . 

A computer program as claimed in any one of claims 3 3 to 
35, wherein the code or instructions for sorting are 
operable to sort the clips into groups based on clip 
classifications . 

A computer program as claimed in any one of claims 22 to 
36 comprising code or instructions for attributing a 
label descriptor to each clip. 



A computer program as claimed in claim 37, wherein the 
code or instructions for attributing are operable to 
automatically attribute a label descriptor to a clip.. 

A computer program as claimed in claim 37, wherein the 
code or instructions for attributing are operable to 
attribute a label descriptor to a clip in response to a 
user selection. 

A computer program as claimed in any one of claims 22 to 

39, wherein the code or instructions for sorting are 
operable to sort the clips into one or a set of folders. 

A computer program as claimed in any one of claims 22 to 

40, wherein the code or instructions for sorting are ** 
operable to include each clip in a plurality of 
different groups. 

A digital camera or the like that includes a system or 
computer program as claimed in any one of the preceding *• 
claims, the digital camera being operable to provide the I 
video footage for editing. 

A method for editing digital video footage using a 
computer processor, the method comprising cutting the 
digital video footage into a plurality of clips and 
sorting the clips into a plurality of groups. 

A method as claimed in claim 43, comprising presenting 
on a screen or display a group identifier for each group 



of clips; receiving a user selection of a desired group 
identifier and presenting clip identifiers for clips 
included in the selected group in response to the user 
selection . 



A method as claimed in claim 43 or claim 44, comprising 
searching for groups of clips based on user input 
criteria . 



A method as claimed in any one of claims 43 to 45, 
wherein the step of cutting is done automatically using 
a cutting algorithm. 

A method as claimed in any one of claims 43 to 46, 
wherein the step of cutting involves detecting changes ' 
between frames of the video and defining a cut when the 
changes exceed a pre -determined level. 

A method as claimed in any one of claims 43 to 47, 
wherein the step of cutting involves cutting the video * 
content into clips on the basis of an intensity edge *• 
determination . 



A method as claimed in any one of claims 43 to 48, 
wherein the step of cutting comprises using a histogram 
algorithm. 



A method as claimed in any one of claims 43 to 49, 
wherein the step of cutting involves receiving a user 
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command and cutting the video content into clips in 
response to the user command. 

A method as claimed in any one of claims 43 to 50, 
comprising manipulating or modifying clips in response 
to a user command. 

A method as claimed in claim 51, wherein the step of 
manipulating comprises merging selected clips in 
response to a user command . 

A method as claimed in claim 51, wherein the step of 
manipulating involves dividing a selected clip in 
response to a user command. 

A method as claimed in any one of claims 43 to 53 
comprising classifying the clips according to pre- 
determined classification criteria. 

A method as claimed in claim 54, wherein the step of 
classifying comprises classifying the clips according to 
any one or more of camera motion detection; optical 
flow; image quality; scene changes; photometric 
characteristics; level of blurring and contrast 
detection . 

A method as claimed in claim 54 or claim 55, wherein the 
step of classifying comprises classifying a clip in 
response to a user defined classification. 



A method as claimed in any one of claims 54 to 56, 
wherein the step of sorting is done based on clip 
classifications . 

A method as claimed in any one of claims 43 to 57 
comprising attributing a label descriptor to each clip. 

A method as claimed in claim 58, wherein the step of 
attributing comprises automatically attribute a label 
descriptor to a clip. 

A method as claimed in claim 58, wherein the step of 
attributing comprises attributing a label descriptor to 
a clip in response to a user selection. 

A method as claimed in any one of claims 43 to 60, 
wherein the step of sorting comprises sorting the clips 
into one or a set of folders. 

A method as claimed in any one of claims 43 to 61, * 
wherein the step of sorting comprises including each I 
clip in a plurality of different groups. 

A method as claimed in any one of claims 43 to 62 
comprising adding clips to the groups. 



A method as claimed in claim 63 wherein the step of 
adding is done automatically. 
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A method as claimed in claim 63 wherein the step of 
adding is done in response to a user command. 

A method as claimed in any one of claims 43 to 65 
comprising creating new groups. 

A method as claimed in claim 66 wherein the step of 
creating is done automatically. 

A method as claimed in claim 67 wherein the step of 
creating is done in response to a user command. 

A method as claimed in any one of claims 43 to 68 
comprising using selected video clips to provide video 
content . 

A method as claimed in claim 69 comprising copying the 
video content. 

Video content that is produced using the system or 
computer program or method of any one of the preceding 
claims . 

Copies of video content that is produced using the 
system or computer program or method of any one of the 
preceding claims. 

A system substantially as described hereinbefore with 
reference to the accompanying drawings. 



A computer program substantially as described 
hereinbefore with reference to the accompanying 
drawings . 

A method substantially as described hereinbefore with 
reference to the accompanying drawings. 
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